Systems and methods for centralized control of autonomous vehicles

ABSTRACT

Disclosed are systems, methods and devices for centralized control of autonomous vehicles. In some embodiments, a system and method allow an autonomous control system on-board an autonomous vehicle to pass control of the autonomous vehicle to an offboard panel of experts upon encountering an anomaly. In some embodiments, a system and method allow a regulatory entity to proactively distribute rules and requirements to autonomous vehicles while operating within a regulated space.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/359,654, which was filed on 7 Jul. 2016, which is incorporated hereinin its entirety by this reference thereto.

FIELD OF THE INVENTION

At least one embodiment of the present invention pertains to autonomousvehicles.

BACKGROUND

Once viewed as a futuristic technological concept that was forever overthe horizon, autonomous vehicles are now widely considered an imminentinevitability. Advances in sensor technologies and control systems havespawned a proliferation of Advanced Driver Assistance Systems (ADAS) incurrent-generation vehicles, including adaptive cruise control (ACC),parking assistance (e.g., automatic parallel parking), blind spotmonitoring and land change assistance, forward collision warning, andlane departure warning.

The even more advanced artificial intelligence systems onboard thenext-generation vehicles currently under development go far beyondassistance, promising fully autonomous operation of the vehicle. Indeedthe sensor and control systems onboard soon-to-be-released autonomousvehicle designs can safely navigate the vast majority of drivingsituations without human input at all.

Yet even the most optimistic of autonomous vehicle designers concedethat substantial challenges remain. In particular, even the mostadvanced autonomous vehicles under development struggle to properlynavigate highly anomalous, less-frequently encountered “edge cases”. Anddespite extensive internal representations of traffic regulations,autonomous vehicles possess a limited ability to adequately respond tospatial and temporal variations in regulatory frameworks.

SUMMARY OF THE INVENTION

Disclosed are systems, methods and devices for centralized control ofautonomous vehicles. In some embodiments, a system and method allow anautonomous control system-on-board an autonomous vehicle to pass controlof the autonomous vehicle to an offboard panel of experts uponencountering an anomaly. In some embodiments, a system and method allowa regulatory entity to proactively distribute rules and requirements toautonomous vehicles, while the autonomous vehicles operate within anregulated space, and/or proactively distribute updated rules andrequirements by which the autonomous control system onboard theautonomous vehicle can operate in one or more regions.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present invention are illustrated by wayof example and not limitation in the figures of the accompanyingdrawings, in which like references indicate similar elements.

FIG. 1 is a flow chart of an illustrative method for resolution ofanomalous situations via a remote panel of experts.

FIG. 2 is a schematic view of an illustrative autonomous vehicle.

FIG. 3 is schematic view of an illustrative system for centralizedcontrol of an autonomous vehicle.

FIG. 4 is a schematic diagram of an illustrative autonomous controlsystem integrated with a vehicle.

FIG. 5 is a schematic diagram of an illustrative autonomous controlsystem that is functionally integrated with an advanced driverassistance system (ADAS).

FIG. 6 shows a centralized system in communication with vehiclesoperating in any of one or more regions and/or regulated spaces.

FIG. 7 is a schematic view of centralized operation of one or morevehicles within a region.

FIG. 8 is a block diagram of illustrative anomalies that can beencountered during autonomous operation of one or more vehicles, whichcan be determined or sensed by any of the autonomous vehicles, or by theexternal entity.

FIG. 9 shows illustrative system architecture configured for an externalpanel of experts.

FIG. 10 shows an illustrative hierarchy between a supervisory expert andone or more advisory experts.

FIG. 11 is a flowchart of an illustrative method showing the wirelessdistribution of information, such as rules, regulations and/orrequirements, from a regulatory entity to one or more autonomousvehicles, and subsequent monitoring.

FIG. 12 is a high-level block diagram showing an example of a processingdevice that can represent any of the systems described herein.

DETAILED DESCRIPTION

References in this description to “an embodiment”. “one embodiment”, orthe like, mean that the particular feature, function, structure orcharacteristic being described is included in at least one embodiment ofthe present invention. Occurrences of such phrases in this specificationdo not necessarily all refer to the same embodiment. On the other hand,the embodiments referred to also are not necessarily mutually exclusive.

Introduced here are methods, systems and devices that allow centralizedcontrol of one or more autonomous vehicles that can be implemented for awide variety of system environments and operating conditions.

In certain embodiments, the method and system can allow for anautonomous control system on-board an autonomous vehicle to pass controlof the autonomous vehicle to an external entity, such as an offboardpanel of experts, under one or more conditions where the autonomousvehicle encounters an anomaly.

In certain embodiments, the method and system can allow a regulatoryentity to proactively distribute rules and requirements to one or moreautonomous vehicles, such as when the autonomous vehicles are operatingin a regulated space.

Resolution of Anomalous Situations Via Remote Panel of Experts

Irregular road signs or lane markings, construction zones, detours, andeven unusually-shaped or colored roadside objects and shadows canpresent substantial challenges to autonomous vehicle systems. Severalautonomous vehicle developers have proposed (ref. “Tesla reveals all thedetails of its Autopilot and its software v7.0”; 2015 Oct. 14;http://electrek.co/2015/10/14/) characterizing and cataloguing suchanomalies, enabling “fleet-wide” learning. But unless these approachescan abstract and internalize lessons to be applied to future anomalies,there will remain a “first time for everything”. Moreover, relativelymore routine anomalies, such as inclement weather, can confuse (e.g.,through changes in color or contrast) if not outright obscure sensingsystems (ref. “The cold, hard truth about autonomous vehicles andweather”, Fortune, 2015 Feb. 2,http://fortune.com/2015/02/02/autonomous-driving-bad-weather/).

Accordingly, it appears there will be a transition period in whichautonomous vehicle systems will provide considerable utility to manydrivers, but will not deliver complete autonomous operation. During thisperiod, autonomous vehicle systems will require at least occasionalinput or guidance from human operators in the form of “criticalinterventions” (ref. “For Now, Self-Driving Cars Still Need Humans”, TheNew York Times, 17 Jan. 2016,http://www.nytimes.com/2016/01/18/technology/driverless-cars-limits-include-human-nature.html/),when the autonomous vehicle encounters an anomaly it cannot reliablyaddress. Consequently, the State of California has proposed regulations(ref. “Deployment of Autonomous Vehicles for Public Operation”,http://dmv.ca.gov/portal/dmv/detail/vr/autonomous/auto) requiring thatautonomous vehicles be operated by a licensed driver who could take overif necessary” (ref. “California D.M.V. Stops Short of Fully EmbracingDriverless Cars”, The New York Times, 16 Dec. 2015,http://www.nytimes.com/2015/12/17/technology/california-dmv-stops-short-of-fully-embracing-driverless-cars.html).

Obligating a human occupant to be prepared to take control of a motorvehicle, on very short notice, greatly diminishes the potential benefitof autonomous vehicles. Indeed, a primary promise of autonomous vehicleproponents has been the ability to free human drivers to deeply engagein more productive tasks while traveling from one location to another.In addition, while autonomous vehicle occupants may be aware of thisobligation, many will remain tempted to engage in other activities andmay be slow in responding to requests to take control. It would thus beadvantageous to provide a system and method that reduces the reliance onvehicle occupants in resolving anomalies encountered by an autonomousvehicle.

As such, some embodiments of the methods and systems disclosed hereinare configured to resolve anomalies encountered by an autonomousvehicle. Upon encountering an anomaly, if possible, the autonomouscontrol system-on-board the autonomous vehicle passes control of theautonomous vehicle to an offboard, i.e., remote panel of experts thatassess and resolve the anomaly. In some embodiments, the offboard panelof experts include one or more human experts. In some embodiments,additional information can be provided to the panel of experts from theautonomous vehicle to enhance the capabilities of the panel of experts.Only if necessary will the autonomous control system pass control of theautonomous vehicle to the driver. The system thus reduces the engagementand effort required of the driver, improving safety and rendering theautonomous vehicle (as perceived by the driver) more fully autonomous.In some embodiments, the panel of experts, upon identifying a knownanomaly, can prompt the passage of control to the driver, or communicatethe information to the autonomous control system that enables theautonomous control system to properly address the anomaly.

FIG. 1 is a flow chart of an illustrative method 10 for resolution ofanomalous situations via a remote panel of experts 235 (FIG. 3).Operation of the system begins 12 when an autonomous control system 110(FIG. 2) associated with an autonomous vehicle 100 (FIG. 2), enters intoa control loop 15. The illustrative autonomous control system 110 seenin FIG. 4 includes a processor 302 (FIG. 4) connected to a memory 304(FIG. 4), and is typically connected to a communication module 305 (FIG.4), such as including a transceiver 306 (FIG. 4) that is connected to anantenna 112 (FIG. 2) for sending and receiving wireless signals 220(FIG. 3).

As further seen in FIG. 1, the autonomous control system 110continuously maintains control 14 of the autonomous vehicle 100, andperiodically monitors 16 the environment and its internal state foranomalies 484, e.g., 484 a-484 k (FIG. 8).

If the method determines 18 that an anomaly 484 is not detected 20, theautonomous control system 110 continues 22 to maintain operation of theautonomous vehicle 100, by implementing control loop 15. If it isdetermined 18 that an anomaly 484 is detected 24, the illustrativeautonomous control system 110 determines 32 if the detected anomaly 484requires prompt attention or resolution. If not 28, the autonomouscontrol system 110 passes control 30 of the autonomous vehicle 100 to anexternal entity 235, e.g., a panel of experts 235, wherein the panel ofexperts 235 can assess and resolve 32 the anomaly 484, and when safe todo so, the method 10 returns control 22 to the autonomous control system110 for operation within the control loop 15.

If it is determined 32 that the detected anomaly 484 requires promptresolution 34, the autonomous control system 110 determines 36 if theautonomous vehicle 100 can be quickly and safely maneuvered to apassively safe state. If so 38, the autonomous control system 110maneuvers 40 the autonomous vehicle 100 to the passively safe state, andthen passes 30 control of the autonomous vehicle 100 to the panel ofexperts 235, to assess and resolve 32 the anomaly 484, and when safe todo so, the method 10 returns control 22 to the autonomous control system110 for operation of the autonomous vehicle 100 within the control loop15.

If the autonomous control system 110 determines 36 that the autonomousvehicle 100 cannot 42 be quickly and safely maneuvered to a passivelysafe state, the autonomous control system 110 passes 44 control of theautonomous vehicle 100 to the driver USR (FIG. 2), at which point thedriver USR can proceed 46 to assess and resolve the specific anomaly484, and when safe to do so, the driver USR can return control 22 to theautonomous control system 110 for operation of the autonomous vehicle100 within the control loop 15.

FIG. 2 is a schematic view 80 of an illustrative autonomous vehicle 100,such as located in a region 402 (FIG. 6) of operation, in which such aregion 402 typically includes one or more roadways 130.

The illustrative autonomous vehicle 100 seen in FIG. 2 includes anautonomous control system 110 that can be linked to onboard systems 308(FIG. 4), such as including the power system 310, i.e., the engine ormotor, the electrical system 312, the braking system 314, the steeringsystem 316, the communications system 318, an offboard and/or onboardglobal position system (GPS) 114, one or more cameras 116, a forwardlooking infrared (FLIR) camera 120, a light detection and ranging(LIDAR) sensor 118, as well as any other vehicle systems.

FIG. 3 is schematic view of an illustrative system 200 for centralizedcontrol of an autonomous vehicle 100. The autonomous vehicle 100 seen inFIG. 2 and FIG. 3 can communicate 220, such as over one or morecommunication networks 225, to send and/or receive information. Forinstance, the illustrative autonomous vehicle 100 seen in FIG. 2 andFIG. 3 can communicate 230 with an offboard panel of experts 235, and insome embodiments can communicate 240 with a regulatory entity 245.

FIG. 4 is a schematic diagram 300 of an illustrative autonomous controlsystem 110 integrated with an autonomous vehicle 100, wherein theautonomous vehicle 100 includes user controls 320 for interaction withone or more vehicle systems 308, and a user interface 322 forinteraction with the autonomous control system 110.

FIG. 5 is a schematic diagram 360 of autonomous functions that can beprovided by an illustrative autonomous control system 110, such as usingone or more functional modules 380, and other determined or storedinformation. For instance, any of maps 376, vehicle specifications 377,or default rules of operation 378 can be stored within the memory 304,while the autonomous control system 110 can include any of roadwaydetection 382, pedestrian detection 384, object detection 386, and/orother detection 388.

In some embodiments, the illustrative autonomous control system 110 canbe functionally integrated with an advanced driver assistance systemfeatures (ADAS) 362, which can include any of adaptive cruise control(ACC) 364, parking assistance (e.g., automatic parallel parking) 366,blind spot monitoring 368, lane change assistance 370, forward collisionwarning 372, and lane departure warning 374.

FIG. 6 is functional view 400 of an illustrative centralized system 200in communication with autonomous 100 vehicles operating in any of one ormore regions 402 and/or regulated spaces 404. FIG. 7 is a schematic view440 of centralized operation of one or more autonomous vehicles 100within at least a portion of an illustrative region 402, such as withrespect to one or more roadways 130.

FIG. 8 is a schematic diagram 480 of illustrative anomalies 484, e.g.,484 a-484 k, that can be encountered during autonomous operation of oneor more autonomous vehicles 100, wherein the anomalies 484 may bedetermined or sensed by any of one or more of the autonomous vehicles100, by the panel of experts 235, or in some circumstances by aregulatory entity 245. The illustrative set 482 of anomalies 484 seen inFIG. 8 that may be encountered by an autonomous vehicle 100 include anyof objects or obstacles 484 a, irregular road signs 484 b, irregularlane markings 484 c, construction zones 484 d, detours 484 e, weather484 f, emergency conditions 484 g, shadows and/or lighting 484 h, orother anomalies 484 k.

Generally, the autonomous control system 110 can be configured to detectone or more anomalies 484, such as when the autonomous vehicle 100 andintegrated autonomous control system 110 encounters environmentalconditions, or attains an internal state under which it can no longerensure, to an acceptable level of certainty, safe control of theautonomous vehicle 100.

For example, in some embodiments, the autonomous control system 110 candetect an outage of one or more sensors or information sources. In anillustrative situation, a LIDAR sensor 118 may be temporarily occluded,or a network outage may prevent receipt, e.g., via communication pathway220, of map information 376 (FIG. 5).

In another circumstance, the autonomous control system 110 can detectinconsistencies between multiple environmental sensors. For example, theautonomous control system 110 may detect an unacceptably low level ofagreement between a LIDAR unit 118 and a forward looking infrared (FLIR)camera 120.

In another scenario, the autonomous control system 110 may detectinconsistencies between direct observations and internal models of theenvironment 130. For instance, the autonomous control system 110 maydetect an unacceptably low level of agreement between the directionexpected for a roadway 130, such as based on an internal map 376 (FIG.5), and a roadway direction as actually determined, such as by a roadwaydetection module 382 (FIG. 5), using an edge detection algorithm,operating on a forward looking video camera 116.

In some operating situations, the autonomous control system 110 maydetect low levels of confidence in determinations made by one or moreits processing modules 380. For example, the roadway detection module382 may report a low level of certainty in its estimate of the far-fieldroadway location, or a pedestrian detection module 384 may report a highdegree of uncertainty in the position of a previously detected andtracked pedestrian.

In some situations, the autonomous control system 110 may detect that aprior action has not had the expected effect. For example, theautonomous control system 110 may determine that an evasive maneuver toavoid a particular detected object 484 a, such as by an object detectionmodule 386 (FIG. 5), has not in fact changed the position of theautonomous vehicle 100 relative to the detected object 484 a, suggestingthat the autonomous control system's understanding of the environment isflawed, and/or that the current, i.e. updated, position of the object484 a requires a further evasive maneuver.

In some embodiments, if an anomaly 484 is detected, the autonomouscontrol system 110 can determine 32 (FIG. 1) whether the anomaly 484requires prompt resolution. For example, the autonomous control system110 can consider the maximum tolerable duration of a particular sensoroutage, or the forward speed of the autonomous vehicle 100, incombination with the distance to a spatial anomaly 484.

If the anomaly 484 requires prompt resolution 34 (FIG. 1), theautonomous control system 110 can then determine 36 whether theautonomous vehicle 100 can be quickly and safely maneuvered 40 to apassively safe state. A passively safe state is one that affords asubstantial period of additional time within which to resolve theanomaly 484. For example, maneuvering 40 to a passively safe state mayinclude a substantial reduction in speed along a straight and clearroadway 130, or a full stop of the autonomous vehicle 100 outside theflow of traffic. If such a maneuver can be performed 38 (FIG. 1), theautonomous control system 110 can perform the maneuver 40 (FIG. 1) tobring the autonomous vehicle 100 to a passively safe state.

After the autonomous vehicle 100 has reached a passively safe state, orif the autonomous control system 110 determines that the anomaly 484does not require 28 prompt resolution, the autonomous control system 110passes control 30 of the autonomous vehicle 100 to an offboard panel ofexperts 235.

FIG. 9 shows illustrative system architecture 500 configured for anexternal panel of experts 500, such as implemented with one or morecomputers. For instance, the illustrative expert system architecture 500seen in FIG. 9 includes a central system node 502, such as including aprocessor 504 and a transceiver 506 for communication 230 over acommunication network 225.

The illustrative central system node 502 seen in FIG. 9 is alsointerconnected to one or more expert nodes 508,512, such as including asupervisory node 508, and one or more advisory nodes 512. Each of thenodes 508, 512 can correspond to an expert, such as a supervisory expertSUP associated with the supervisory node 508, and advisory experts ADVassociated with corresponding advisory nodes 512. The illustrativesupervisory node 508 seen in FIG. 9 includes a corresponding userinterface 510 for interaction with the supervisory expert SUP, while theillustrative advisory nodes 512 each include a corresponding userinterface 514 for interaction with the advisory experts ADV.

FIG. 10 shows an illustrative hierarchy 540 between a supervisory expertSUP at a supervisory node 508, and one or more advisory experts ADV atcorresponding advisory nodes ADV. As such, in some embodiments, theoffboard panel of experts 235 can comprise one or more humans SUP, ADV(FIG. 10) at a centralized, remote location. One or more members of thepanel of experts 235 may be assigned to resolve the detected anomaly.

As seen in FIG. 10, the panel of experts 235 can include a hierarchy540, such as between an advisory node 512, e.g., 512 a, wherein anadvisor ADV, such as having associated expertise 542, analyzes one ormore operational factors, and a supervisory node 508, in which asupervisor SUP has a higher decision making authority over one or moreadvisory experts ADV. In some embodiments, a decision from the panel ofexperts 235 is based on a vote 544 between a plurality of experts508,512, such as input within a time threshold, which can also beweighted based on the expertise 542 and hierarchy of the experts 508,512.

In some embodiments, the autonomous control system 110 passes as muchinformation as possible to the offboard panel of experts 235. Forexample, the autonomous control system 110 may indicate the specificanomaly 484 detected, and the one or more reasons the anomaly 484 wasdetected. In some embodiments, the autonomous control system 110 canalso provide the feed signal of a live camera, e.g., 116, as well asimagery prior to anomaly detection. In some embodiments, the autonomouscontrol system 110 can also provide additional raw data (e.g., lightlevel, temperature, or humidity readings) or interpreted sensorinformation (e.g., depth maps and roadway or pedestrian detections). Insome embodiments, relevant information from other sources can be used tosupplement information 520 (FIG. 9) received from the autonomous controlsignal, such as weather information, traffic information, informationreceived from other autonomous vehicles 100 in the same area, othercamera feeds from the area, updated road conditions, and/or emergencyinformation.

The offboard panel of experts 235 can resolve the anomaly 484 using oneor more of several possible approaches. For example, under somecircumstances, the offboard panel of experts 235 may immediatelydetermine that there is in fact no anomaly 484. In some situations, theoffboard panel of experts 235 can advise the autonomous control system110 to ignore raw or interpreted sensor information that the offboardpanel of experts 235 determines to be spurious. In other situations, theoffboard panel of experts 235 can instruct the autonomous vehicle 100 tomaneuver itself 40 (FIG. 1) to a passively safe state. Given sufficientonboard sensor capability (e.g., video feeds) and bandwidth between theautonomous vehicle 100 and the offboard panel of experts 235, in someembodiments, the offboard panel of experts 235 can remotely operate theautonomous vehicle 100 for an extended period of time.

After adequately assessing and fully resolving an anomaly 484 using oneor more of these or other approaches, the offboard panel of experts 235can pass control 22 of the autonomous vehicle 100 back to the autonomouscontrol system 110. Alternatively, if the offboard panel of experts 235determines that the anomaly 484 cannot be successfully addressedremotely, it may request assistance from the driver USR of theautonomous vehicle 100 (such as through the user interface 322 (FIG.4)), or pass control of the autonomous vehicle 100 to the driver USR.Under such circumstances, some embodiments of the system 200 and method10 can notify the driver USR of the situation, such as through auditoryand or visual alerts through the user interface 322, and in someembodiments can provide automated or human-based communication with thedriver USR.

In some situations, the system 200 and method 10 can notify the driverUSR of a pending situation 484, e.g., through user interface 322 (FIG.4), in which the driver USR may soon need to assume control of theautonomous vehicle 100, such as during analysis of the anomaly 484 bythe panel of experts 235, but before a decision is reached by the panel235. In such a scenario, the driver USR or other passengers may benotified that supplementary information may be requested by the expertpanel 235, such as related to the current anomaly 484, or the currentstatus of the driver USR.

Detailed Examples

Operation of the method and corresponding system as outlined in FIG. 1can be better understood through consideration of detailed examples.

In a first example, an autonomous vehicle 100 under control of theautonomous control system 110 traveling a residential street 130encounters a body of water 484 a (FIG. 8) spanning the width of theroadway 130. In this example, the autonomous control system 110 detectsan anomaly 484 a when a path planning module 390 (FIG. 5), based on ananalysis of the roadway roughness and visual textures, reports a lowlevel of confidence that the proposed path is safely traversable.Because the autonomous vehicle 100 is rapidly approaching the body ofwater 484 a, the autonomous control system 110 determines 32 that theanomaly 484 a requires prompt resolution 34. Based on low levels ofsurrounding traffic and the presence of an open parking lane alongsidethe roadway 130, the autonomous control system 110 determines 36 thatthe autonomous vehicle 100 can quickly and safely maneuver 40 to apassively safe state. The autonomous control system 110 maneuvers 40 theautonomous vehicle 100 to a standing position within the parking lane.The autonomous control system 110 then passes 30 control of theautonomous vehicle 100 to the offboard panel of experts 235, provides alive video feed, and indicates the proposed (but deemed uncertain) paththrough the body of water 484 a. In some embodiments, the offboard panelof experts 235 can watch other vehicles pass through the body of water,and conclude it is merely a shallow puddle. As such, in this example,the offboard panel of experts 235 can transmit instructions to theautonomous control system 110 to proceed on the proposed path, andpasses control 22 of the autonomous vehicle 100 back to the autonomouscontrol system 110. In some embodiments, the offboard panel of experts235 can consider relevant information regarding the capabilities and/orlimitations, e.g., vehicle specifications 337 (FIG. 5) of the autonomousvehicle 100, or any difference between the autonomous vehicle 100 andother vehicles, e.g., ground clearance, vehicle weight, vehicle size,available power, air intake height, etc.). In this manner, the system200 may result in a decision that is appropriate for the specificautonomous vehicle 100. For example, even if it is observed that a4-wheel drive truck was able to successfully navigate the body of water484 a, a specific smaller autonomous vehicle 100 may not be so able, andthe offboard panel of experts 235 can respond accordingly.

In another example, an autonomous vehicle 100 under control of theautonomous control system 110 and traveling a remote, rural highway 130,passes through a swarm of insects. Debris from impacted insects coatsand occludes an aperture of the LIDAR 118 unit atop the autonomousvehicle 100. The autonomous control system 110 detects an anomaly 484when it determines that the depth map from the LIDAR 118 is compromised.The autonomous control system 110 detects no nearby traffic and theroute ahead is straight and on well-maintained roads. The autonomouscontrol system 110 determines that the autonomous vehicle 100 can safelyproceed for a substantial distance relying solely upon a lane detectionalgorithm analyzing textures and lane markings as seen via onboard videocameras, e.g., 116. The autonomous vehicle 100 can therefore be directedto proceed onward while the autonomous control system 110 passes controlto the offboard panel of experts 235. The autonomous control system 110provides the offboard panel of experts 235 with a live video feed andthe depth map that was determined to be compromised. Based on the depthmap (or lack thereof), the offboard panel of experts 235 concludes thatthat the LIDAR unit 118 has been completely occluded. The offboard panelof experts 235 can safely guide the autonomous vehicle 100 to a parkedposition alongside the roadway 130, and can instruct the driver USA ofthe autonomous vehicle 100 to visually inspect the LIDAR unit 118. Ifand when the driver USR or other person, e.g., another passenger or aservice station attendant, clears the insect debris from the LIDAR unit118, the offboard panel of experts 235 can receive an improved depthmap. The offboard panel of experts 235 can then declare the anomaly 484to be resolved, and pass control 22 of the autonomous vehicle 100 backto the autonomous control system 110.

Alternative Embodiments

In various alternative embodiments, the autonomous control system 110can bypass the determining 36 if the autonomous vehicle 100 can bequickly and safely maneuvered to a passively safe state. Instead, if theanomaly 484 requires prompt resolution, the autonomous control system110 can directly pass control to the driver USR. In some embodiments,determining 36 if the passively safe state is safely and quicklyattainable under control of the autonomous control system 110 ispreferable, however, in that it reduces the number of anomalies 484 thatwill ultimately require assistance from the driver USR of the autonomousvehicle 100.

In other alternative embodiments of the method 10 and system 200, theautonomous control system 110 can consider one or more time thresholdswhile awaiting resolution of the anomaly 484 by the offboard panel ofexperts 235. For example, if the autonomous control system 110determines 32 that the anomaly 484 does not 28 require promptresolution, it may afford the offboard panel of experts 235 only a briefperiod of time before either (a) attempting to maneuver 40 theautonomous vehicle 100 to a passively safe state or (b) passing 44control to the driver USR. If the autonomous control system 110determines 32 that the anomaly 484 does require 34 prompt resolution andmaneuvered 40 the autonomous vehicle 100 to a passively safe state, thesystem 200 may afford the offboard panel 235 a relatively longer periodof time before passing 44 control to the driver USR. In either case,considering the time thresholds ensures more reliable operation in theevent of a communications network delay or outage.

As described above, the monitoring 16 for anomalies 484, as seen in FIG.1, can be performed by the autonomous control system 110 based strictlyon locally sensed information and the internal state. In somealternative embodiments, the autonomous control system 110 can beexplicitly informed of an anomaly 484 from an exterior informationsource 520, e.g., (FIG. 9) or authority, e.g., an appropriate regulatoryagency 245. The authority information source 520 or regulatory agency245 may inform the autonomous vehicle 100 of the anomalies 484 via anyof the methods of Centralized Distribution of Vehicle Imperatives, suchas described below.

For example, in some embodiments, an external navigation informationsource 520 can describe specific regions 402 (e.g., mountain passes ornarrow tunnels) in which the autonomous control system 110 is known tobe incapable of reliably operating the autonomous vehicle 100. Upon orjust prior to entering into such regions 402, the autonomous controlsystem 110 can immediately detect or predict such an anomaly 484. Insome system embodiments 200, the definition of these regions 402 canalso include an instruction to the autonomous control system 110,indicating whether control should be passed 30 to the offboard panel ofexperts 235 or passed 44 to the driver USR.

In another example, the autonomous control system 110 may be informed ofanomalous conditions 484 by a local information source 520 or authority245 via a wireless communications channel 220. For example, upon entryinto a parking facility, the operator (e.g., a human operator or anautomated operator) of the parking facility may inform the autonomouscontrol system 110 of an anomaly 484 (i.e., challenging parkingconditions) and request that the autonomous control system 110 passcontrol to a local offboard panel of experts 235. In this case, theoffboard panel of experts 235 can be a set of one or more remoteoperators, i.e., “harbor masters”, that are tasked with efficiently andsafely parking autonomous vehicles 100 within the parking garage.Government authorities 520,245 may similarly broadcast informationregarding anomalies 484 to all autonomous vehicles 100 within a region402 if unusual traffic patterns or safety hazards exist. As discussedbelow, in some embodiments, external information 520 can be communicatedto the autonomous vehicle 100 in regard to a localized anomaly 484,e.g., a parking garage, an airport road network, etc., wherein theautonomous control system 110 is provided with localized information 520with which to operate, e.g., a localized map, available parking spaces,controlled movement around an airport for ingress, passenger departure,passenger pickup, egress, etc.

Centralized Distribution of Vehicle Imperatives.

Traffic laws are not consistent across the United States or eventhroughout a single state. For example, turning right on a red light ispermitted in California (ref. “California Driver Handbook—Turns”,California Department of Motor Vehicles, 2016 May 16,https://www.dmv.ca.gov/portal/dmv/detail/pubs/hdbk/turns), while it isprohibited in New York City (ref. “Right Turn on Red”, New York CityDepartment of Transportation, 2009,http://www.nyc.gov/html/dot/downloads/pdf/ssi09_rightonred.pdf).Similarly, speed limits and many other requirements are not consistentacross government boundaries.

Regulatory agencies, e.g., 245, also frequently have vehiclerequirements that are temporary. Some common examples of this include,limited lanes during construction time periods, school zone speedlimits, or no parking on street cleaning days. Other one-time examplesalso exist, such as redirecting traffic during a particular event, orredirecting traffic around an accident.

Some illustrative embodiments of the systems and methods disclosedherein allow a regulatory entity 245 (FIG. 3) to push rules andrequirements to autonomous vehicles 100, which the autonomous vehicles100 are required to obey while operating within a correspondingregulated space 404.

FIG. 11 is a flowchart of an illustrative method showing the wirelessdistribution of information, such as rules, regulations and/orrequirements, from a regulatory entity 245 to one or more autonomousvehicles 100, and subsequent monitoring 610 and/or actions 616.

In some embodiments, a regulatory entity 245 can be a government entitysuch as a state, county, city, or other municipality that has theauthority and desire to regulate autonomous vehicles 100. The regulatoryentity 245 has authority to define rules within its regulatory region402 or space 404. In some embodiments, a regulated space 404 can bedefined by a geographic boundary or by a fence or property line. In suchcases, the regulatory entity 245 may be the property owner or theirproxy.

In some illustrative embodiments the present invention, a regulatoryentity 245 defines rules, e.g., 392 (FIG. 5) that apply to autonomousvehicles 100 that operate within the regulated space 404. When anautonomous vehicle 100 crosses into the regulated space, such asmonitored 602 (FIG. 11), the autonomous vehicle 100 downloads the rules392, such as received 604 from the relevant regulatory agency 245, andfollows them 606 until it leaves the regulated space 404. Whenever anautonomous vehicle 100 is outside such a regulated space 404, theautonomous vehicle 100 is free to act on its own again, i.e., withoutbeing limited by the specific rules 392 that apply to the regulatedspace 404.

Some embodiments of such a method 600 and system 200 can be implementedin a manner similar to that of the Federal Aviation Administrationsystem of traffic control. The autonomous vehicle 100 can beinterrogated by the system 200 when it enters a regulated space 404,which allows for the regulatory entity 245 to ensure that the propertaxes or tolls have been paid, and that the rules are being followed.Some embodiments of such a system and method can be implemented toinstitute other regulations, such as weight or mileage fees, and/oralerts associated with any of vehicle condition, registration and/oremissions.

In some embodiments, these rules are distributed on a government onlyfrequency or channel, such as any of a secure or private communicationchannel that does not permit public access, with security andencryption. While a communication channel or frequency that is separatefrom cellular internet isn't a requirement, it makes the system morerobust against attacks, and adds a secondary communication channel 220,which can be disintermediated from the cellular network, e.g., 225. Thiscan allow the system 200 to use national level encryption standards, toprovide any of anti-jam capable, frequency agile, low probability ofintercept, low probability of detection, low probability of jamming, andanti-spoofing technologies. In some embodiments, such a system can scanall of the known frequencies until it finds one that is being used.

The use of a separate frequency also allows for some embodiments 600 tobe used as a civil event notification system. For instance, otherelectronic devices can monitor 610 the frequency for a specialannouncement, such as in the case of an emergency, e.g., an approachingaccident, or an amber alert, which may cause the electronic device 616to instruct a person on how to respond to the emergency. In an exemplaryembodiment, an alert associated with a local crime or nearby suspect mayallow interaction 616 with autonomous vehicles 100 and/or theiroccupants, to allow the live feeds of cameras, e.g., 116, to beaccessed, recorded, and/or stored.

In some embodiments, other devices can also monitor 610 and broadcast onthis network as well. For instance, in an illustrative embodiment,traffic and parking control devices can report data back to thecentralized system 200, e.g., 235,245, for use in routing autonomousvehicles 100, or to be used to identify parking locations.

In some embodiments, upon entering a regulated space 404, an autonomousvehicle 100 can also receive a set of fall back or default rules, e.g.,378 (FIG. 5), to be used in case the communication system becomesunavailable. These commands 378 could instruct the autonomous vehicle100 to follow the basic rules of the road. In some embodiments, the fallback or default rules 378 can also instruct all non-moving autonomousvehicles 100 to wait for a short period of time before embarking withouta functional communication link 220.

Some embodiments of the system 200 and methods 10,600 can be fundedthrough a number of different methods. In one instance, the system 200can be paid for by levying a tax on any electronic device that utilizesthe relevant frequencies. Alternatively, some embodiments of the system200 and methods 10,600 can be funded by requiring payment to accessrestaurant, navigational, and traffic data 520. In some embodiments, thesystem 200 and methods 10,600 can request payment in order to give anautonomous vehicle 100 priority to get to a location faster.

The operation of the system 200 and methods 10,600 may be furtherunderstood using the following additional examples:

In one example, the system 200 may issue a command prohibitingnon-autonomous driving in certain areas. For example, the diamond laneon the freeway may only permit autonomous carpools during certain times.This would allow tight packs, or platoons, of autonomous vehicles 100 toquickly travel down the freeway 130. In another example, a curfew couldbe established that prohibited non-autonomous driving past a certainhour of the night for safety.

In another instance, the system 200 and methods 10,600 can route trafficaround an accident 484. In many cases, the alternative roadways 130 thatare used to route vehicles around an accident are not capable ofcarrying the same number of vehicles. Some embodiments of such a system200 and methods 10,600 can allow for intelligent routing, wherein eachalternative route is allocated a certain percentage of the detouredvehicles 100, to decrease the likelihood of traffic jams on thealternative routes 130.

In another illustrative embodiment, traffic control during specialevents or construction can also become significantly easier. Such anembodiment can allow for a local traffic control officer to issueinstructions to autonomous vehicles 100, regardless of what trafficlights or signs may indicate or say. For example, an officer can directan autonomous vehicle 100 to travel through a red light.

In the case of a dangerous situation 484 such as a nuclear meltdown, ahurricane, a tornado, an earthquake, a police action, or a militaryoperation, some embodiments of the system 200 and methods 10,600 canabandon certain driving norms to allow for an increased flow of traffic.For example, the width of lanes can be decreased to allow a four lanefreeway to carry six lanes of traffic. Driving on the shoulder may alsobe permitted. In a worst case scenario, some embodiments of the system200 and methods 10,600 can allow vehicles to drive onto a sidewalk, orforce parked vehicles 100 off of the roadway 130, to make room for moretraffic lanes, or to provide increased access for emergency vehicles.

In some embodiments, the autonomous vehicles 100 can be directed todrive in a pattern, such as a staggered pattern, in which the autonomousvehicles 100 can retain the ability to navigate between lanes, such asfor ingress/egress, and/or merging. Such a pattern would also be helpfulfor integrating non-autonomous vehicles (e.g., cars/trucks/motorcyclesand/or emergency vehicles), such that non-autonomous vehicles can mergein and out as needed.

In some embodiments, street cleaning can also be made significantlyeasier, as autonomous cars 100 parked in the path of a street cleanercan be commanded to move either to another location, or to drive aroundthe block while the street cleaning is done.

High speed chases would also become significantly safer, as police coulddirect autonomous cars 100 ahead of the chase to exit the freeway, orpull to the side of the road 130.

In some embodiments, one or more property owners or managers can operateas regulatory entities 245 with regard to their correspondingproperties. For example, the regulatory entity 245 may be a parkinggarage owner or a sports stadium. This local regulatory entity 245transmits rules 392 to the autonomous vehicle 100 detailing how theautonomous vehicle is to operate on the property. These rules 392 maycontain instructions on where to park, removing the need to havenumerous people directing traffic at a sports stadium.

In some embodiments, the regulatory entity 245 can also direct theautonomous vehicle 100 to park in a manner that would otherwise notpermit a vehicle to exit, such as parking vehicles 3-4 deep. The owneror driver of the autonomous vehicle 100 would then notify the regulatoryentity 245 before they plan to leave the property. This would allow theregulatory entity 245 to instruct the various autonomous vehicles 100 toshuffle around to allow access to the owners autonomous vehicle 100.Alternatively, the property may track the autonomous vehicle owner'ssmartphone, and move the autonomous vehicle 100 when it detects that theowner is in the process of leaving the property.

FIG. 12 is a high-level block diagram showing an example of a processingdevice 900 that can be a part of any of the systems described above,such as the autonomous control system 110, the expert system 500 and/ornode 502, the supervisory node 508, advisory nodes 512, or a systemassociated with regulatory entity 245. Any of these systems may be orinclude two or more processing devices such as represented in FIG. 12,which may be coupled to each other via a network or multiple networks.In some embodiments, the illustrative processing device 900 seen in FIG.12 can be embodied as a machine in the example form of a computer systemwithin which a set of instructions for causing the machine to performone or more of the methodologies discussed herein may be executed.

In the illustrated embodiment, the processing system 900 includes one ormore processors 902, memory 904, a communication device and/or networkadapter 906, and one or more storage devices and/or input/output (I/O)devices 908, all coupled to each other through an interconnect 910. Theinterconnect 910 may be or include one or more conductive traces, buses,point-to-point connections, controllers, adapters and/or otherconventional connection devices. The processor(s) 902 may be or include,for example, one or more general-purpose programmable microprocessors,microcontrollers, application specific integrated circuits (ASICs),programmable gate arrays, or the like, or a combination of such devices.The processor(s) 902 control the overall operation of the processingdevice 900. Memory 904 may be or include one or more physical storagedevices, which may be in the form of random access memory (RAM),read-only memory (ROM) (which may be erasable and programmable), flashmemory, miniature hard disk drive, or other suitable type of storagedevice, or a combination of such devices. Memory 904 may store data andinstructions that configure the processor(s) 605 to execute operationsin accordance with the techniques described above. The communicationdevice 906 may be or include, for example, an Ethernet adapter, cablemodem, Wi-Fi adapter, cellular transceiver, Bluetooth transceiver, orthe like, or a combination thereof. Depending on the specific nature andpurpose of the processing device 900, the I/O devices 908 can includedevices such as a display (which may be a touch screen display), audiospeaker, keyboard, mouse or other pointing device, microphone, camera,etc.

Unless contrary to physical possibility, it is envisioned that (i) themethods/steps described above may be performed in any sequence and/or inany combination, and that (ii) the components of respective embodimentsmay be combined in any manner.

The autonomous control system and corresponding methods introduced abovecan be implemented by programmable circuitry programmed/configured bysoftware and/or firmware, or entirely by special-purpose circuitry, orby a combination of such forms. Such special-purpose circuitry (if any)can be in the form of, for example, one or more application-specificintegrated circuits (ASICs), programmable logic devices (PLDs),field-programmable gate arrays (FPGAs), etc.

Software or firmware to implement the techniques introduced here may bestored on a machine-readable storage medium and may be executed by oneor more general-purpose or special-purpose programmable microprocessors.A “machine-readable medium”, as the term is used herein, includes anymechanism that can store information in a form accessible by a machine(a machine may be, for example, a computer, network device, cellularphone, personal digital assistant (PDA), manufacturing tool, or anydevice with one or more processors, etc.). For example, amachine-accessible medium includes recordable/non-recordable media,e.g., read-only memory (ROM); random access memory (RAM); magnetic diskstorage media; optical storage media; flash memory devices; etc.

Those skilled in the art will appreciate that actual data structuresused to store this information may differ from the figures and/or tablesshown, in that they, for example, may be organized in a differentmanner; may contain more or less information than shown; may becompressed, scrambled and/or encrypted; etc.

Note that any and all of the embodiments described above can be combinedwith each other, except to the extent that it may be stated otherwiseabove or to the extent that any such embodiments might be mutuallyexclusive in function and/or structure.

Although the present invention has been described with reference tospecific exemplary embodiments, it will be recognized that the inventionis not limited to the embodiments described, but can be practiced withmodification and alteration within the spirit and scope of the examplesdisclosed herein. Accordingly, the specification and drawings are to beregarded in an illustrative sense rather than a restrictive sense.

What is claimed is:
 1. A method implemented with an autonomous controlsystem that includes a processor and a memory, the method comprising:autonomously operating a vehicle; monitoring an internal state of thevehicle and an environment in which the vehicle is operating; based onthe monitoring, upon detecting an anomaly, determining if it is safe topass control of the vehicle to an external entity; and passing thecontrol of the vehicle to the external entity when it is determined tobe safe.
 2. The method of claim 1, wherein the anomaly includes any ofan environmental condition or an attained internal state of the vehicleunder which the autonomous control system cannot ensure safe control ofthe vehicle within a predetermined level of uncertainty.
 3. The methodof claim 1, further comprising: once the anomaly is resolved by theexternal entity, resuming autonomous operation of the vehicle.
 4. Themethod of claim 1, wherein the external entity includes a panel of oneor more experts.
 5. The method of claim 1, wherein the passing controlof the vehicle to the external entity is determined to be safe when theanomaly does not require resolution within a predetermined period oftime.
 6. The method of claim 1, wherein the passing control of thevehicle to the external entity is determined to be safe after theautonomous control system maneuvers the vehicle to a passively safestate.
 7. The method of claim 1, further comprising: passing control toa driver of the vehicle to resolve the anomaly when it is determinedthat it is not safe to pass control over to the external entity.
 8. Themethod of claim 7, further comprising: once the anomaly is resolved bythe driver, resuming autonomous operation of the vehicle.
 9. The methodof claim 1, further comprising: receiving information from a regulatoryentity; and operating the vehicle in any of a region or space based onthe received information.
 10. The method of claim 9, wherein thereceived information includes any of a rule or regulation for operationof the vehicle in the region or space.
 11. The method of claim 9,further comprising: storing the received information in the memory foroperation of the vehicle in the region or space.
 12. A system forautonomous control of a vehicle, comprising: a wireless transceiver; amechanism for monitoring the operation of the vehicle; a mechanism formonitoring the environment in which the vehicle is operating; anautonomous system controller for controlling the operation of thevehicle within the environment, based on the monitored operation of thevehicle and the monitored environment, wherein the autonomous systemcontroller includes a processor and a memory, wherein the processorincludes instructions for performing a method comprising: autonomouslyoperating the vehicle; monitoring the operation of the vehicle and theenvironment in which the vehicle is operating; based on the monitoring,upon detecting an anomaly, determining if it is safe to pass controlover to an external entity; and with the wireless transceiver over awireless network, passing the control of the vehicle to the externalentity when it is determined to be safe.
 13. The system of claim 12,wherein the mechanism for monitored operation of the vehicle includesany of operation of a power system, an electrical system, a brakingsystem, a steering system, and a communications system.
 14. The systemof claim 12, wherein the mechanism for monitoring the environment inwhich the vehicle is operating includes a sensor system.
 15. The systemof claim 14, wherein the sensor system includes any of a forward lookinginfrared (FLIR) device, a light detection and ranging (LIAR) sensor, andone or more cameras.
 16. The system of claim 12, wherein the anomalyincludes any of an environmental condition or an attained internal stateof the vehicle under which the autonomous control system cannot ensuresafe control of the vehicle within a predetermined level of uncertainty.17. The system of claim 12, wherein the processor includes instructionsfor: resuming autonomous operation of the vehicle, once the anomaly isresolved by the external entity.
 18. The system of claim 12, wherein theexternal entity includes a panel of one or more experts.
 19. The systemof claim 12, wherein the passing control of the vehicle to the externalentity is determined to be safe when the anomaly does not requireresolution within a predetermined period of time.
 20. The system ofclaim 12, wherein the passing control of the vehicle to the externalentity is determined to be safe after the autonomous control systemmaneuvers the vehicle to a passively safe state.
 21. The system of claim12, wherein the processor includes instructions for: passing control toa driver of the vehicle to resolve the anomaly when it is determinedthat it is not safe to pass control over to the external entity.
 22. Thesystem of claim 21, wherein the processor includes instructions for:resuming autonomous operation of the vehicle once the anomaly isresolved by the driver.
 23. The system of claim 12, wherein theprocessor includes instructions for: receiving information from aregulatory entity over a wireless network; and operating the vehicle inany of a region or space based on the received information.
 24. Thesystem of claim 23, wherein the received information includes any of arule or regulation.
 25. The system of claim 23, wherein the processorincludes instructions for: storing the received information in thememory for the operating of the vehicle in the region or space.
 26. Amethod for operating a vehicle over a wireless communication networkwith an external system that includes a processor, the methodcomprising: receiving information at the external system over thewireless communication network from an autonomous control systemintegral with the vehicle, wherein the received information includes ananomaly detected by the autonomous control system, wherein the anomalyis based on information monitored by the autonomous control systemregarding any of an internal state of the vehicle and the environment inwhich the vehicle is operating; remotely operating the vehicle over thewireless communication network, while: assessing the anomaly based onthe received information, and resolving the anomaly based on theassessment; and returning the control of the vehicle to the autonomouscontrol system when the anomaly is resolved.
 27. The method of claim 26,wherein the anomaly includes any of an environmental condition or anattained internal state of the vehicle under which the autonomouscontrol system cannot ensure safe control of the vehicle within apredetermined level of uncertainty.
 28. A system for operating one ormore autonomous vehicles, the method comprising: a transceiver forcommunication with the autonomous vehicles over a wireless communicationnetwork; a communication link to an expert system; a processor linked tothe transceiver and to the communication link, wherein the processorincludes instructions for: receiving information over the wirelesscommunication network from an autonomous control system integral with aparticular autonomous vehicle of the autonomous vehicles, wherein thereceived information includes an anomaly detected by the autonomouscontrol system, wherein the anomaly is based on information monitored bythe autonomous control system regarding any of the internal state of theparticular autonomous vehicle and the environment in which theparticular autonomous vehicle is operating; over the communication link:passing control of the particular autonomous vehicle to the externalsystem; and transmitting information regarding the anomaly to theexternal system, wherein the external entity can: assess the anomalybased on the received information, and resolve the anomaly based on theassessment; and over the wireless communication network, returning thecontrol of the particular autonomous vehicle to the autonomous controlsystem when the anomaly is resolved.
 29. The system of claim 28, whereinthe anomaly includes any of an environmental condition or an attainedinternal state of the vehicle under which the autonomous control systemcannot ensure safe control of the vehicle within a predetermined levelof uncertainty.
 30. A method implemented by a central authority,comprising: transmitting information over a wireless communicationchannel from the central authority to an autonomous vehicle regardingoperation of the autonomous vehicle within any of a region or space;wherein the autonomous vehicle includes an autonomous system controllerassociated therewith; and wherein subsequent operation of the autonomousvehicle within the region or space is based on compliance with thetransmitted information.
 31. The method of claim 30, wherein thetransmitted information includes any of a rule or regulation foroperation of the autonomous vehicle within the region or space.
 32. Themethod of claim 30, wherein the transmitted information is required tobe locally stored for and accessible by the autonomous system controllerfor operation of the autonomous vehicle within the region or space. 33.The method of claim 30, wherein the transmitted information ensures thatthe autonomous vehicle has paid a regulatory fee for operation withinthe region or space.
 34. The method of claim 30, further comprising:transmitting a fallback set of rules over the wireless communicationchannel from the central authority to the autonomous vehicle regardingoperation of the autonomous vehicle within the region or space; whereinsubsequent operation of the autonomous vehicle within the region orspace is based on compliance with the fallback set of rules if thecentral authority cannot be reached over the wireless communicationchannel.
 35. The method of claim 30, wherein the communication channelis any of a secure or private communication channel.
 36. The method ofclaim 30, further comprising: receiving information at the centralauthority regarding operation of the autonomous vehicle within theregion or space.
 37. The method of claim 36, further comprising:monitoring the communication channel for a condition related to any ofan environment, or the space or region in which the autonomous vehicleis operating; and taking an action to interact with any of theautonomous system controller or an occupant of the autonomous vehiclebased on a monitored condition.
 38. The method of claim 37, wherein themonitored condition is an approaching accident.
 39. The method of claim37, wherein the action includes any of issuing a special announcement,declaring an emergency, or issuing an amber alert.
 40. The method ofclaim 37, wherein the action includes instructions on how to respond tothe condition.
 41. The method of claim 37, wherein the action includesestablishing remote access to a live feed of a camera associated withthe autonomous vehicle, for any of viewing, recording, or storing thelive feed.